Methods for visual identification of cognitive disorders

ABSTRACT

A method and system for generating a classifier to classify facial images for cognitive disorder in humans. The system comprises receiving a labeled dataset including set of facial images, wherein each of the facial image is labeled depending on whether it represents a cognitive disorder condition; extracting, from each facial image in the set of facial images, at least one learning facial feature indicative of a cognitive disorder; and feeding the extracted facial features into a to produce a machine learning trained model to generate a classifier, wherein the classifier is generated and ready when the trained model includes enough facial features processed by a machine learning model.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional application and claims the benefit ofU.S. Non-Provisional application Ser. No. 16/880,683, now allowed, filedon May 21, 2020, the contents of which are hereby incorporated byreference.

TECHNICAL FIELD

The present disclosure relates generally to machine vision techniques.

BACKGROUND

Dementia is one of the most common diseases in the world, with hundredsof millions of diagnosed patients expected in thirty years. As longevityextends average lifespans, and as medical diagnostics advance, rates ofprevalence and detection of dementia increase with time. Dementia posesa significant burden on the afflicted and their caretakers. Manydementia patients would benefit from earlier detection, as would theircaretakers, who would be better able to prepare for the challenges ofcaring for a person afflicted with dementia.

Dementia refers to a broad class of neurological conditions affecting apatient's ability to think, recall, and function. The most common causeof dementia is Alzheimer's disease, followed by vascular conditions, theformation of Lewy bodies, and the degeneration of various components ofthe frontal and temporal lobes of the brain. Currently, no known curefor dementia exists. Certain modes of treatment, including medication,behavioral therapy, music therapy, and diet and exercise modification,are employed to offset the symptoms of dementia or to otherwise improvequality of life for the patient and his or her caretaker. With earlierdetection, certain precautions may be taken to ameliorate the effects ofdementia.

The prevalence of dementia contributes to the detriment suffered, notonly by caretakers, patients, and friends and family members, but bycommunities and nations as a whole. By some estimates, 2011 projectionsof 4.5 million dementia patients in the United States may rise as highas 13.8 million patients in the United States and 130 million patientsglobally by 2050. This prevalence of dementia presents medical, social,and economic ramifications, and may require governments to react to thechallenges of providing adequate medical and welfare services to theafflicted. As early detection may allow for improved treatment andmitigation, improved methods for reliable dementia diagnosis may bedesirable.

Current methods of detecting dementia center around the application ofabilities tests such as the mini mental state examination (MMSE), theabbreviated mental test score (AMTS), the modified mini-mental stateexamination (3MS), the cognitive abilities screening instrument (CASI),the trail-making test, the clock drawing test, and the Montrealcognitive assessment (MoCA). Further, these cognitive assessmentsrequire physicians' attention and a non-negligible amount of time, arenot dependent on any one dementia-specific hallmark and, as such, canyield inaccurate results when cognitive conditions other than dementiaare present. As a result, such diagnostics are well-suited to detectionof manifest dementia, but are not applicable for early detection oflarge numbers of patients.

As an example of the tests considered for the diagnosis of dementia, theMMSE is the most common. The MMSE includes eleven tasks and questions,from which an examinee must reach a score of twenty-four out of thirtypoints to avoid a dementia diagnosis. For an experienced physician,administering the test may take at least seven minutes. The MMSE, asapplied, presents specific weaknesses which may hinder early detectionof dementia. In review of the MMSE, studies have indicated 87% testsensitivity and 82% specificity. Another study presented furtherfindings of a 21% missed diagnosis rate and a 20% false-positivediagnosis rate in a hospital setting. In addition, there are conflictingresults between MMSE and other methods in the detection of early-stagedementia cases. Further, the MMSE includes a bias in favor of patientlearning variables, skewing results and preventing compete, accurate,and reliable diagnosis. As a result, improved methods of detectingdementia may be useful to the treatment of the condition.

Further, alternative dementia diagnoses may reduce or eliminate the needfor in-person doctor visits and evaluation. By streamlining thediagnostic process to include remote, image-based diagnosis, patiente-diagnosis may reduce the need for a patient to visit a doctor, for thedoctor to address patients individually, and for insurance companies andother payors to pay for a full consultation where a routine diagnosismay suffice. Promoting the application of remote and telemedicine in thefield of dementia diagnosis may allow for reduced costs for patients andpayors, reduced time investments from both patients and physicians, andimprovements in diagnostic efficiency over in-person consultation. Inaddition, the use of telemedicine techniques may be well-suited topatients who are unable to visit a doctor due to the patient'scondition, remote location, or susceptibility to certain communicablediseases. As a result, patients who would ordinarily be unable toreceive consultation and treatment may be treated more effectively, andthe risk of disease transmission between patients and physicians may bereduced or eliminated.

In addition, alternative diagnostic techniques may be applicable to thediagnosis of other conditions presenting characteristic symptoms. Asconditions such as Parkinson's, Alzheimer's, anxiety, depression, andothers continue to afflict millions of patients globally, developmentand application of alternative diagnosis may provide for identificationand treatment of non-dementia conditions and may allow patients,practitioners, and healthcare payors to enjoy the benefits describedherein.

It would, therefore, be advantageous to provide a solution that wouldovercome the challenges noted above.

SUMMARY

A summary of several example embodiments of the disclosure follows. Thissummary is provided for the convenience of the reader to provide a basicunderstanding of such embodiments and does not wholly define the breadthof the disclosure. This summary is not an extensive overview of allcontemplated embodiments and is intended to neither identify key orcritical elements of all embodiments nor to delineate the scope of anyor all aspects. Its sole purpose is to present some concepts of one ormore embodiments in a simplified form as a prelude to the more detaileddescription that is presented later. For convenience, the term “someembodiments” or “certain embodiments” may be used herein to refer to asingle embodiment or multiple embodiments of the disclosure.

Certain embodiments disclosed herein also include a method forgenerating a classifier to classify facial images for cognitive disorderin humans. The method comprises receiving a labeled dataset includingset of facial images, wherein each of the facial image is labeleddepending on whether it represents a cognitive disorder condition;extracting, from each facial image in the set of facial images, at leastone learning facial feature indicative of a cognitive disorder; andfeeding the extracted facial features into a to produce a machinelearning trained model to generate a classifier, wherein the classifieris generated and ready when the trained model includes enough facialfeatures processed by a machine learning model.

Certain embodiments disclosed herein also include a system forgenerating a classifier to classify facial images for cognitive disorderin humans. The system comprises a processing circuitry; and a memory,the memory containing instructions that, when executed by the processingcircuitry, configure the system to: receive a labeled dataset includingset of facial images, wherein each of the facial image is labeleddepending on whether it represents a cognitive disorder condition;extract, from each facial image in the set of facial images, at leastone learning facial feature indicative of a cognitive disorder; and feedthe extracted facial features into a to produce a machine learningtrained model to generate a classifier, wherein the classifier isgenerated and ready when the trained model includes enough facialfeatures processed by a machine learning model.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter disclosed herein is particularly pointed out anddistinctly claimed in the claims at the conclusion of the specification.The foregoing and other objects, features, and advantages of thedisclosed embodiments will be apparent from the following detaileddescription taken in conjunction with the accompanying drawings.

FIG. 1 is a diagram utilized to describe the various embodiments.

FIG. 2 is an example flow diagram illustrating the generation andapplication of a supervised model used for classifying a facial image,according to an embodiment.

FIG. 3 is a flowchart illustrating the training of a learning model fordetection of dementia based on facial expression.

FIG. 4 is a flowchart illustrating the application of the variousembodiments to diagnosis of disease from facial data.

FIG. 5 is a diagram illustrating a sample architectural layout for aconvolutional neural network (CNN), according to an embodiment.

FIG. 6 shows a block diagram of the system implemented according to anembodiment.

FIGS. 7A, 7B, and 7C are example images applicable to describe thedisclosed embodiments.

DETAILED DESCRIPTION

It is important to note that the embodiments disclosed herein are onlyexamples of the many advantageous uses of the innovative teachingsherein. In general, statements made in the specification of the presentapplication do not necessarily limit any of the various claimedembodiments. Moreover, some statements may apply to some inventivefeatures but not to others. In general, unless otherwise indicated,singular elements may be in plural and vice versa with no loss ofgenerality. In the drawings, like numerals refer to like parts throughseveral views.

FIG. 1 is an example diagram 100 utilized to describe the variousembodiments. The diagram 100 depicts a data source 110, a diagnosissystem 130, and an optional data store 140 communicating over a network120. The network 120 may be, but is not limited to, a wireless,cellular, or wired network, a local area network (LAN), a wide areanetwork (WAN), a metro area network (MAN), the Internet, the world wideweb (WWW), a network similar to those described, and any combinationthereof.

In an example embodiment, the data source 110 may be, but is not limitedto, a set of facial images, the set including one or more facial images,a streamed video or videos, a data feed from a personal computer, a cellphone, PDA, a surveillance system camera or camcorder, or the like, or aphysician's terminal or other device specifically configured for theembodiment. The data stored in each data source 110 may be in a formatof structured data, unstructured data, semi-structured data, or acombination thereof.

The diagnosis system 130 (or, simply, the “system” 130) is configured toperform various embodiments disclosed herein. Specifically, the system130 is configured to implement processes for isolating facial data inimages and video. As noted above, all images and video (collectivelyreferred to as “images”) received from the data source 110 are processedto isolate facial data. In an embodiment, to allow the isolation offacial data, the system 130 may implement one or more machine learningtechniques.

According to the disclosed embodiments, the system 130 may be configuredto identify facial features in images. To this end, the system 130 maybe configured to identify facial features and expressions in imagesusing neural networks such as, as an example and without limitation, aconvolutional neural network (CNN) such as that described with respectto FIG. 5 , below.

The system 130 may be utilized to identify and index facial data inimages stored in, or otherwise provided by, the data sources 110. Tothis end, the system 130 is configured to generate a flexible structureto support differently formatted data inputs that represent differenttypes of facial data from the data source 110. In yet anotherembodiment, the system 130 may be configured to support parallelprocessing of facial data and, more specifically, parallel analysis offacial data from one or more data sources.

As will be discussed in detail below, the system 130 is configured toimplement a process for identifying and categorizing facial data.Actions can be performed in real-time on the received data using a setof predefined commands. In an alternate embodiment, the system 130 maybe realized as a separate device such as a local computer, otherprocessing hardware, as a cloud-based distributed service, or adistributed resource.

The system 130 may be implemented as a physical machine, a virtualmachine, or a combination thereof. A block diagram of an exampledepicting a physical machine implementation is discussed below withreference to FIG. 6 . A virtual machine may be any virtual softwareentity, such as a software container, a micro service, a hypervisor, andthe like.

The data store 140 may be a relational database or a NoSQL type ofdatabase such as, but not limited to, MongoDB, ElasticSearch, and thelike. Examples of relational databases may include, but are not limitedto, Oracle®, Sybase®, Microsoft SQL Server®, Access®, Ingres®, and thelike. In an embodiment, the data store 140 may be a plurality of logicalentities residing in the same physical structure. In someconfigurations, the data store 140 is optional.

According to an embodiment, the data store 140 may be configured tohouse diagnostic facial data. The diagnostic facial data housed in thedata store 140 may include, but is not limited to, data extracted fromfacial recognition analyses, diagnostic data tags, such as whether thedata extracted from facial recognition analyses was extracted from aface belonging to a person with a condition such as dementia, or acorrelative factor expressing the degree to which the diagnostic datatag is accurate for a given extracted facial recognition analysis dataset.

According to an embodiment, the data store 140 may store diagnosticfacial data accounting for age progression. In an embodiment, the facialdiagnostic data housed in the data store 140 may include data featuresaccounting for age progression including, but not limited to, multiplefacial information data sets for one individual, with each data settagged with the date of each profile data set and a time-specificdiagnostic data tag indicating whether the disease or condition waspresent at the time the facial information data set was housed in thedata store 140.

In an embodiment, the optional data store 140 may be included in thesystem 130. In an alternate embodiment, the optional data store 140 maybe realized as separate components connected directly with the network120, with the system 130, or with both the system 130 and the network120. Further, the optional data store 140 may be removed or excludedfrom various configurations and embodiments as needed.

In an embodiment, the system 130 may be configured to assess thelikelihood that facial data, received from the data source 110 via thenetwork 120, contains indicators of disease, including indicators ofdementia. In an embodiment, the system 130 may be further configured toassess the likelihood of disease by extracting, from the facial data setreceived from the data source 110 via the network 120, a standardizedfacial data profile, according the system 130. In an embodiment, thesystem 130 may be configured to assess the likelihood of disease bycomparing the standardized facial data profile extracted by the system130 with standardized facial data profiles received from the data store140 via the network 120.

It should be noted that the embodiments disclosed herein are not limitedto the specific architecture illustrated in FIG. 1 , and that otherarchitectures may be equally used without departing from the scope ofthe disclosed embodiments. Specifically, the system 130 may reside in acloud computing platform, a datacenter, and the like. The cloudcomputing platform may be a private cloud, a public cloud, a hybridcloud, and the like. Moreover, in an embodiment, there may be aplurality of systems operating as a distributed system. Further, thedata store 140 may be distributed as well. In some implementations, thesystem 130 may be an internal component or instance of any of the datasources 110. In an embodiment, the system 130 may include one or moredata stores 140, configured to house diagnostic facial data as well asother, like, data.

FIG. 2 is an example flow diagram 200 illustrating the generation andapplication of a supervised model used for classifying a facial image,according to an embodiment. For sake of simplicity and withoutlimitation of the disclosed embodiments, FIG. 2 will be discussed alsowith reference to the elements shown in FIG. 1 .

The framework depicted in the diagram 200 operates in two phases:learning and detection. In the learning phase, a trained model 201 isgenerated and trained and, in the detection phase, the trained model 201is utilized for detection of dementia in images of people. Such imagesare received from the data source 110 or from other, similar sources.The trained model 201, in another embodiment, is generated and trainedusing supervised or semi-supervised machine learning techniques. Imagesmay include video, pictures, video frames, live video feed, and thelike.

Training and detection may, in an embodiment, occur in cycles. Where adetected facial image is returned to a client, and where feedbackreceived from client, indicating whether the detection results arecorrect or incorrect, the image may be added to the learning dataset,and the trained model 201 may be refined accordingly.

In the learning phase, images including at least one human face arereceived from sources such as the data source 110 and other sources. Thereceived images are aggregated and saved as a learning dataset 210. Theaggregation of images may be for a predefined time window (e.g., allimages received from the data source 110 during a one-hour time window)or as specified by a user, operator, or predefined scheduling program.

The learning dataset 210 may include images of patients with confirmeddementia diagnoses, patients confirmed to be dementia-free, patientswith unknown diagnoses, and any combination thereof, further includingthe patients' MMSE scores or other indicators of dementia status.Further, the learning dataset 210 may include images of patients sampledat various periods, providing age-progression data for patientsappearing in images taken or recorded across multiple time periods.Example age-progression images are shown with respect to FIGS. 7Athrough 7C, below. The inclusion of patient aging examples, and theinclusion of samples from patients with varying dementia diagnoses, mayallow subsequent analysis of dementia diagnoses in additional patients,and may allow for differentiation between the visible effects ofdementia and the visible effects of aging.

The learning dataset 210 is input into a pre-processor 220 configured tofilter out “noisy” inputs. The pre-processor 220 may be configured tonormalize or otherwise adjust images for evaluation and comparison. Thepre-processor 220 may standardize the images received from the learningdataset by adjusting image features such as, as examples and withoutlimitation, size, resolution, compression, color, other, like, features,and any combination thereof.

The preprocessed images are fed into a feature extractor 230 configuredto extract a set of features representing a human face. In anembodiment, the feature extractor 230 may be configured to identifyfacial features including, as examples, and without limitation, eyes,lips, cheeks, noses, faces generally, and other, like, features. Facialfeatures may be identified and extracted using techniques from thedomain of facial recognition computing, such as landmark analysis,whereby features are identified based on geometry, i.e. recognizing anellipse-like shape as an eye, or based on comparison to sample ortemplate faces. The extraction of facial features by the featureextractor 230 may include some or all of the processes, methods,elements, or aspects described with respect to S330 of FIG. 3 , below.

In an embodiment, the feature extractor 230 may be configured toautomatically detect and identify additional facial features in anyimage included in the dataset. Those features may be extractedexplicitly or as part of the model training phase.

In an additional embodiment the feature extractor 230 may apply geneticalgorithm methods, wherein a first facial feature, such as the eyes, aredetected, and the remaining feature regions, including those regionsadjacent to or containing the first facial feature, are generated basedon a predefined genetic algorithm.

The facial features identified and extracted by the feature extractor230 may allow for the training of a learning model based on the featuresextracted. In addition to the extracted features, relationships betweenthe features extracted may provide relevant diagnostic data and mayallow for the recognition of common hallmark features in subsequentanalyses. As examples, relationships determined based on the extractedfeatures may include the distance between two features, the angle of agiven feature relative to a given centerline, the overall size or shapeof the face identified, other, like, relationships, and any combinationthereof.

In an embodiment, multiple features may be extracted from a given imageand may be analyzed to provide diagnostic information, to train alearning system, or both. Further, diagnostic accuracy and learningsystem training accuracy may be enhanced by the extraction of multiplefeatures from multiple face images included in the learning dataset 210.

The extracted features, reflecting the size, shape, and positioning ofvarious aspects of faces, are fed to the neural network 240. In anembodiment, the neural network 240 may be a deep neural network and mayimplement a semi-supervised or a supervised machine learning algorithmfor analyzing the features and generating, using external supportingdata, a trained model 201. Further, the neural network may be combinedwith, supplemented by, or entirely replaced with other, suitable machinelearning algorithms, techniques, and methods. The trained model 201 mayprovide one or more classification rules or rulesets applicable to thecreation of the classifier 280. The classifier 280 may be generated andconfigured as “ready” where the trained model 201 includes a volume ofextracted facial features sufficient to provide consistent, accuratediagnosis. Further, one or more classifiers may be generated including,without limitation, multiple classifiers generated to correspond to, andwhich may be configured for the diagnosis of, unique types of cognitivedisorders. Examples of the semi-supervised machine learning algorithmsinclude, and the algorithm may be based on, deep neural networks,support vector machines (SVM), decision trees, label propagation, localoutlier factor, isolation forest, and the like.

The supporting data includes labels of certain features previouslyidentified as facial features, information pertaining to a patient'sdiagnosis status, information pertaining to the collection of images, orfacial features, and other, like, information. For example, supportingdata may include labels indicating a patient's known dementia status,the date on which a given image was taken or recorded, and metadatapertaining to the camera and lighting conditions used to capture theface from which the features are extracted.

The trained model 201, generated by the neural network 240 (or thesupervised machine learning algorithm), is used to map or correlatefacial images or their features, or both, to labels indicating a binarydementia status, a dementia status probability, the years elapsedbetween images of the same patient, and other, like, information. Thedetected facial features are analyzed during the detection phase.

According to an embodiment the trained model 201 is generated when thelearning dataset 210 labels are at rest and in a controlled environment,such as in a lab. It should be noted that the trained model 201 may begenerated and trained based on information received from a plurality ofco-located learning dataset 210 labels.

In an embodiment, during the detection phase, images received from thedata source 110 are aggregated and saved as a detection dataset 250. Theaggregation may be performed as described with respect to the learningdataset 210, above.

The detection dataset 250 is input to the pre-processor 260 whichperforms preprocessing similar or identical to that applied with respectto the pre-processor 220, as described above. The pre-processeddatasets, including facial images from the detection dataset 250, arefed into a feature extractor 270. The feature extraction may includesome or all of the aspects or elements of the feature extractionperformed with respect to the learning dataset 210 at the featureextractor 230, above.

The set of features extracted at the feature extractor 270 are fed tothe classifier 280. The classifier 280 is configured to label thefeatures extracted to provide diagnostic information. The labeling isbased on the trained model 201, the facial image, and the featuresextracted from the images analyzed. In addition to labeling images basedon the trained model 201, the classifier 280 may be configured to outputone or more diagnostic scores. The diagnostic scores produced by theclassifier 280 may indicate the presence of a disease or condition, thelikelihood of presence of a disease or condition, and other, like,diagnostic information or diagnostic scores.

The classifier 280 may be configured to output diagnostic scores as informats including, without limitation, binary results, indicatingwhether a disease or condition is present via a “yes or no” data point,numerical or percentage results, indicating the likelihood that adisease or condition is present, in other, like, formats, and anycombination thereof. The classifier 280 can be implemented using knownclassifying techniques utilized in supervised or semi-supervised machinelearning. For example, the classifier 280 can be implemented as a neuralnetwork, a k-nearest neighbors (KNN) model, a gaussian mixture model(GMM), a random forest, manifold learning, decision trees, and the like.The classifier 280 utilizes the trained model 201 generated by theneural network 240. The classifier and the neural network are based onthe same machine learning model.

In an embodiment, the labels provided by the classifier 280 may be fedinto the neural network 240 to improve or otherwise update the trainedmodel 201.

In an example embodiment, any of the neural network 240 and theclassifier 280 can be realized by one or more hardware logic componentsutilized to process artificial intelligence (AI) tasks, such as adigital signal processors (DSPs), tensor processing units (TPUs), orother AI accelerators.

The datasets 210 and 250 can be stored in a memory, which can bevolatile (e.g., RAM, etc.) memory, non-volatile (e.g., ROM, flashmemory, etc.) memory, or a combination thereof. Alternatively, orcollectively, the datasets 210 and 250 can be stored in a storage, orany other medium which can be used to store the desired information.

FIG. 3 is an example flowchart 300 illustrating the training of alearning model for detection of dementia based on facial expression. Alearning model may be deployed to prepare a system for the execution ofa task. By training a learning model on validated, vetted, or otherwiseprepared datasets, the system, including the learning model, may beconfigured to execute a task including analysis of a similar datasetwith minimal or no supervision.

In an embodiment, the method described with respect to the flowchart 300may be applied by a feature extractor and neural network, such as thefeature extractor 230 and the neural network 240 of FIG. 2 , above, totrain a diagnostic model, such as the trained model 201 of FIG. 2 ,above.

In an embodiment, at least one learning dataset is first received atS310. The received learning dataset may include one or more imagescontaining human faces, as well as data labels associated with theimages. The received learning dataset may include images of human facesother than the face of the patient for which diagnosis is sought. In anembodiment, the at least one learning dataset necessarily includes afacial image or images from which a facial data profile may beextracted, a data label identifying the subject of the image or images,and a diagnosis label identifying the diseases or conditions which thesubject of the image or images is known to possess. Further, thereceived learning dataset may include facial images and correspondinglabels identifying whether the labeled facial image represents acognitive disorder condition.

In another embodiment, the at least one learning dataset may alsoinclude, but is not limited to, a data label identifying the date of theimage or images, an age label identifying the age of the subject of theimage or images, and the like.

In yet another embodiment, the at least one learning dataset may bereceived from a physician, physician's assistant, nurse, or othermedical practitioner connected with the diagnosis of the subject of theimage or images. The learning dataset may be similar, or identical, tothe learning dataset 210 of FIG. 2 , above.

Subsequently, in an embodiment, the learning dataset may be analyzed tonormalize received facial images and identify the contents of theincluded data labels, using pre-processing methods, at S320. Theapplication of pre-processing techniques at S320 may includenormalizing, correcting, or otherwise standardizing the received facialimages to reduce or eliminate visual “noise,” permitting subsequentfeature extraction from the normalized facial images. The application ofpre-processing at S320 may include techniques similar or identical tothose applied by the pre-processor, 220, of FIG. 2 , above.

In an embodiment, the application of pre-processing techniques at S320may include the identification of the contents of the included datalabels by separating data labels from the received facial data set. Inan embodiment, pre-processing analysis of the received dataset mayinclude, but is not limited to, the creation of a unique subject profileor the modification of an existing subject profile.

In an embodiment, pre-processing analysis of the included data labelsmay include matching the at least one received dataset to existing ornewly-created subject profiles based on the contents of data labelsincluding, but not limited to, subject name, social security number, orother unique identifiers.

Next, in an embodiment, the at least one received learning dataset maybe analyzed to extract one or more facial features at S330. Theextraction of one or more facial features at S330 may include theextraction of at least one learning facial feature indicative of acognitive disorder. In an embodiment, the facial image or imagesreceived as a part of the at least one learning dataset may be analyzedto render the image or images as a combination of numerical valuesreflecting attributes of the face shown in the image or images, allowingthe extraction of the relevant facial feature data from the datadescribing the face. The techniques used to convert the received imageor images into numerical descriptors may include, and are not limitedto, recognition of facial geometry by analysis of the size, positions,or shape of facial features including the nose, eyes, cheekbones, andjaw. Furthermore, the techniques used to convert the received image orimages into numerical descriptors may also include, but are not limitedto, analysis of 3-D images or profiles and recognition and tracking ofskin features such as lines, wrinkles, and spots. In addition, thetechniques used to convert the received image or images into numericaldescriptors may also include, but are not limited to, processing ofthermal imaging profiles to identify unique features of the subject'sface detectable in thermal spectra. Further, at S330, automatic featureextraction techniques, including landmark analysis, genetic algorithms,and other, like, techniques, including those described with respect tothe feature extractor 230 of FIG. 2 , above, may be applied to extractfacial features.

In an embodiment, the extraction of facial features at S330 may includethe application of one or more convolutional neural networks (CNNs), asdescribed with respect to FIG. 5 , below, to identify and detect facialexpressions and features or include any other supervised methods offeature extraction.

In an embodiment, the methods used to convert the image or imagesincluded in the received at least one learning dataset into numericaldescriptors may include techniques enabling the analysis of facialexpressions. The techniques enabling the analysis of facial expressionsmay include, but are not limited to, tracking of geometric features,such as the nose, chin, cheekbones, and eyes, or the tracking of surfacefeatures such as lines, ridges, and spots, or tension of muscles orskin.

In an embodiment, the methods used to convert the image or imagesincluded in the received learning dataset into numerical descriptors mayinclude techniques for the generation of numerical descriptions of thedifferences arising between images of the same subject at differenttimes, such as those images shown with respect to FIGS. 7A through 7C,below. For a received dataset containing images of a subject, where aprevious dataset contained images of the same subject, and where thecurrently received at least one dataset and the previous dataset containdata tags specifying a different date or subject age, the image orimages received may be compared with the previous image or images.

In an embodiment, the current and previous image or images may becompared by application of the same techniques for numerical descriptionof facial features. Where a difference in the numerical descriptions offacial attributes arises between the current and previous image orimages, the difference in numerical descriptions may be recorded. Wherethe current or previous image or images is or are associated with a datatag specifying a diagnosed disease or condition, the recorded differencebetween the numerical descriptors of facial attributes for the currentand previous image or images may be applicable to diagnosis of similardiseases or conditions from age-progressed images. In the example ageprogression images depicted with respect to FIGS. 7A-7C below, a notablechange in facial expression, and the acuity and focus of the gaze, maybe observed across the depicted time progression. Where such changesindicate the progression of a specific disease or condition over time,the relevant age-progressed facial features may be extracted andgrouped, categorized, or otherwise organized to retain age-progressioninformation during subsequent processing and analysis.

Subsequently, at S340, the received facial dataset may be archived, andthe extracted facial features and associated data labels may be indexed.In an embodiment, the extracted facial features, including generatednumerical descriptions of facial attributes, and the associated datatags and optional time-progression profiles may be indexed to arepository for algorithmic and administrative access. In an embodiment,the extracted facial features, including generated numericaldescriptions, optional time-progression profiles, and separated datalabels, may be indexed for future access. In an embodiment, theextracted facial features may be optimized for storage prior to indexingby techniques including, but not limited to, compression, hashing, andthe like. In an embodiment, the extracted facial features may be indexedaccording to attributes including, but not limited to, numericaldescriptors, diagnosis and condition data tags, and the like.

At the optional step, S350, aggregate diagnostic data profiles, as maybe included in the trained model, 201, of FIG. 2 , above, are generated.As each of the at least one received facial datasets contains a facialimage or images, as well as one or more data labels identifying thesubject and any of the subject's diagnosed diseases or conditions, thecreation of aggregate diagnostic data profiles is possible. In anembodiment, a feature-diagnosis diagnostic data profile may becontinuously updated by recomputing the average numerical descriptors offacial features or diagnoses for a given disease or condition uponindexing extracted facial features associated with a data labelspecifying the given disease or condition. For example, upon indexingextracted facial features associated with a dementia diagnosis data tag,an aggregate diagnostic data profile may be recomputed by taking variousstatistical measures including, but not limited to, the mean, themedian, the mode, and the like, of the numerical descriptors of eachextracted facial feature or set of features tagged with the disease orcondition, including the received extracted facial features.

In an embodiment, a feature-probability aggregate diagnostic dataprofile may be continuously updated by recomputing the likelihood of agiven diagnosis based on numerical descriptors of facial features orexpressions upon indexing extracted facial features. For example, uponindexing extracted facial features associated with a diagnosis data tag,the numerical descriptors of the subject's facial features orexpressions may be compared with the numerical descriptors of otherextracted facial data profiles to identify numerical descriptors commonto multiple extracted facial features associated with the same diseaseor condition data label. For example, if eighty percent of subjectsdiagnosed with a condition have a chin-to-eye separation of more thantwice the length of the nose, newly-indexed extracted facial featuresfitting the same criteria may be matched with an eighty percentlikelihood of diagnosis for that disease or condition. Further, in theexample, the calculation of likelihood of diagnosis based on extractednumerical descriptors may be continuously updated by factoringnewly-indexed extracted facial features, the associated numericaldescriptors, and the associated data labels into the determinativecriteria.

At S360, one or more trained models are generated. The one or moretrained models may be generated in a manner similar or identical to thatdescribed with respect to the trained model, 201, of FIG. 2 , above.Further, the generation of one or more trained models may include theapplication of neural network techniques, including deep learning, whichmay be similar or identical to those described with respect to theneural network, 240, of FIG. 2 , above.

FIG. 4 is an example flowchart 400 illustrating the application of thevarious embodiments to diagnosis of disease from facial data. Suchdiagnosis, as depicted in the flowchart 400, may occur in the featureextractor 270 and classifier 280 of FIG. 2 , above. Diagnosis accordingto the method described in the flowchart 400 may be applicable todiagnosis of cognitive disorders including, without limitation, dementiaand other, like, conditions. At the outset, in the embodiment, detectiondata is received at S410 over a network including, without limitation,the network 120 of FIG. 1 , above. The received detection data, whichmay be, without limitation, captured or uploaded at the data source,110, of FIG. 1 , above. The detection dataset includes at least onestill image containing a human face. The detection data received mayinclude views of multiple faces, views of the same face from multipleangles, views of the same face in different lightings, or views of thesame face obstructed by articles such as, but not limited to, hats,scarves, sunglasses, and the like. Further, the detection dataset may besimilar, or identical, to a detection dataset 250 of FIG. 2 , above.

At S420, the detection data received at S410 is pre-processed andanalyzed to reduce visual noise and detect faces within the detectiondata. In an embodiment, the system 130 accomplishes the pre-processingof detection data by one or more of several methods. The pre-processingmethods applied at S420 may include one or more aspects or elementsdiscussed with respect to the pre-processor, 260, of FIG. 2 , above,including pre-processing methods applicable to the reduction of visualnoise by normalizing, correcting, or otherwise standardizing facialimages in preparation for subsequent feature extraction.

The methods employed in the diagnostic system 130 to pre-processdetection data and detect faces in the received detection data include,but are not limited to, convolutional neural network applications, asdescribed with respect to FIG. 5 , below, other, like, methods, and anycombination thereof. In an embodiment, the accuracy of the algorithmapproach may be further improved by normalization of the detected faceto correct for uneven lighting and head movements. Finally, in anembodiment, the accuracy of the algorithm approach may be improved byselection of detected faces which possess a high degree of fit incomparison to a set of eigenfaces.

At S430, facial features are extracted from the detection data receivedat S410 based on the facial features detected at S420. The isolatedfacial features express the characteristics of the received facial dataas an image-independent dataset and may include, but are not limited to,extracted facial expressions, features, and combinations thereof. Theextracted facial features may include one or more facial featuresindicating cognitive decline or disorder including, without limitation,eyes, lips, cheeks, and other, like, features. The extraction of facialfeatures may be achieved by methods including, but not limited to,geometric methods which assess relative facial feature size andplacement, statistical methods which compare facial images withtemplates, three-dimensional analysis, skin-texture mapping, the like,and any combination thereof.

In an embodiment, the extraction of a facial data profile includes thegeneration of at least one numerical feature describing facialexpression. Further, the extraction of facial features at S430 mayinclude some or all aspects, elements, or sub-processes of the facialdata extraction step S330 of FIG. 3 , above. In addition, the extractionof facial features at S430 may include aspects of the identification andextraction of age-progression facial features, as may be observed withrespect to FIGS. 7A through 7C, below, where the extraction oftime-progressed facial features is further described with respect toS330 of FIG. 3 , above.

At the optional S440, extracted facial features from S430 are comparedwith sample facial features. The sample facial features are housed inthe data store 140 and may contain, without limitation, facial featureinformation of the same format as the facial features extracted at S430,a data label indicating the date on which the sample facial data profilewas generated, a data label indicating the presence of one or morediseases or conditions, and the like. In an embodiment, the comparisonof facial features may include the selection of sample facial featureswhich most closely match the extracted facial features, the selection ofa sample facial feature which most closely matches a specific attributeor attributes of the extracted facial features, or one or more samplefacial features which closely match the extracted facial features basedon overall similarity, attribute similarity, or a combination thereof.In an embodiment, the sample facial features may include some or all ofthe features extracted by the feature extractor 230 of FIG. 2 , above,upon extraction of features from the learning dataset 210, also of FIG.2 , above.

In an embodiment, the comparison of extracted facial features withsample facial features may include the construction of eigenfaces fromsample facial features and the comparison of extracted facial featureswith the set of eigenfaces. As the construction of eigenfaces andsubsequent expression of extracted facial features in terms of theconstructed eigenfaces reduces the processing required versus comparingextracted facial features with individual sample facial features, theconstruction of eigenfaces, and subsequent comparison, may result inimproved processing speed in an embodiment.

At S450, the classification of a disease or condition may be determined.The classification at S450 may include all, some, or none of the outputsgenerated at the optional S440. Where the optional S440 is omitted orincluded in part, classification at S450 may include, but is not limitedto, the identification of sample facial features matching the extractedfacial features, or certain attributes thereof, and the aggregation ofthe disease or condition data labels for each matching facial feature orset of features. The classification of diseases or conditions mayinclude mapping one or more facial features to a score indicating astage of cognitive decline. Further, the classification of diseases orconditions may include assigning one or more scores to one or moreextracted facial features, the scores reflecting the current conditionof a patient's condition including, without limitation, the progressionof a dementia diagnosis, such as between early and advanced dementia. Inan embodiment, an extracted facial feature or set of facial features maybe classified where a certain percentage of those compared sample facialfeatures include data labels for a certain disease or condition. Forexample, if ninety percent of top-matched facial feature sets include adementia label, a classification including a dementia diagnosis, as wellas a confidence rating for the classification, may be established. Theclassification of an extracted facial feature or set of facial featuresas relevant to a disease or condition at S450 may include the generationof a binary “yes or no” classification, a classification certaintyscore, and the like. In an embodiment, the classification, at S450, ofan extracted facial feature or set of facial features relevant to adisease or condition may include features, elements, or aspects similaror identical to those described with respect to the operation of theclassifier, 280, of FIG. 2 , above. Further, classification of anextracted facial feature or set of facial features at S450 may includethe determination of a positive diagnosis of cognitive decline based onone or more scores generated by a classifier, such as the classifier,280, of FIG. 2 , above, upon classification of comparison results. Theclassifier, such as the classifier 280, of FIG. 2 , above, may begenerated as described with respect to FIG. 2 , above, including byfeeding sufficient extracted facial features into a deep neural network,producing the trained model, 201, also of FIG. 2 , above.

In an embodiment, the results generated during the classification atS450, including scores, diagnoses, classifications, and the like, may befed into a neural network, such as the neural network, 240, of FIG. 2 ,above, for application in the training or refinement of a trained model,such as the trained model, 201, also of FIG. 2 , above.

At S460, the classification and confidence rating determined at S450 maybe reported. The reporting of the classification determined at S450 mayinclude, but is not limited to, the condition identified, the confidencedetermined at S450, an identifier tag enabling administrators to verifyresults, and a list of specialists practicing in areas concerning thecondition identified. In an embodiment, the notification may be reportedby means including, but not limited to, e-mail, SMS text message, webbrowser notifications, direct display to the data source 110, directreport to a physician, and other like means.

FIG. 5 is a diagram 500 illustrating a sample architectural layout for aconvolutional neural network (CNN), according to an embodiment. The CNNmay be configured to identify and detect facial features and expressionsin a received input data feature, such as images and additional surfacefeatures, as described above. The CNN may be configured to acceptpreprocessed face images including, for example and without limitation,in a form that is normalized to a two-dimensional matrix and otherfeatures. In an embodiment, the two-dimensional matrix to which thereceived face image or images are normalized may have a size of, forexample, 4096 pixels by 4096 pixels, allowing for capture of fineelements of facial expressions.

The CNN may include multiple convolutional layers 510. Convolutionallayers 510 may be configured to reduce multi-dimensional data featuresinto lower-dimensional filtered data maps or other structures. Theconvolutional layers 510 may process a multi-dimensional data feature bypassing a set of filters, alternatively known as kernels, through thespan of the data feature, recording the interactions between the filtersand the aspects of the data feature as activation maps. In recording theinteraction between the filters and the aspects of the data feature, aCNN may learn filters which activate on detection of specific aspects ofa data feature.

As each filter applied in the convolutional layer 510 results in aseparate activation map, activation maps may be combined for insightinto the data feature as a whole. Activation maps may be “stacked” orcombined to produce an output volume for the convolutional layer, whichmay be subsequently processed by similar methods. The output volume mayhave a similar or identical structure, dimensionality, or both as thereceived data feature, or the output volume may differ from the receiveddata feature in one or both respects.

As an example of the application of a convolutional layer 510, athree-dimensional data feature, visualized as a cube, may be considered.In the example, the individual filters, visualized as planes the size ofa face of the cube, may pass through the cube along the length, therebyfully interacting with every aspect contained within the data feature,and recording the interactions to an activation map. After all thefilters pass through the cube, the resulting two-dimensional activationmaps may be stacked to produce an output volume describing the datafeature based on its interactions with the filters.

In an embodiment, the CNN may be a multi-layered CNN, including a numberof layers greater than or equal to four, and including a mix offully-connected and max-pooling layers 520. In an embodiment, the numberof layers included in the CNN, and the associated hyperparameter tuning,may be set or specified with reference to a sample or hold-out dataset.Hyperparameter tuning may describe the modification of thehyperparameters which determine the size and character of the outputvolume of the convolutional layer 510 as well as dropout rate and otherconfiguration methods. In an embodiment, the hyperparameters modifiedmay include factors affecting connectivity between layered maps whichare stacked to form the output volume, factors affecting construction ofthe output volume in its various dimensions, factors affecting the edgesor borders of the output volume, other, like, factors, and anycombination thereof.

The CNN may include one or more convolutional layers 510, one or moremax-pooling layers 520, other, like, layers, and any combinationthereof. Max-pooling layers 520 may reduce the complexity of a receiveddata feature by dividing the received feature into sub-units, each ofwhich may be clustered within the data feature. The max-pooling layer520 may reduce the complexity of the received data feature by selectingthe highest-valued sub-units in each cluster while preserving thearrangement and relationships between the clusters. The selection of thehighest-valued sub-units in each cluster may be achieved by noting ametric of interest, by which each sub-unit may be evaluated. Theapplication of max-pooling layers 520 provides for a simplification ofthe data features processed, enabling greater subsequent processingspeeds and reduced processing requirements, and allowing a system toidentify and express the important aspects of the received data feature,localized within the feature to preserve significance with respect toother aspects.

In an example, a picture composed of pixels may be divided into a numberof clusters, each including multiple pixels. After assigning a colorscoring scheme which provides values for each color with, as an example,red being the highest and blue being the lowest, a max-pooling layer maysimplify the original image by selecting the highest-scored pixel ineach cluster and noting that pixel as having the maximum value for thatcluster. By this method, the “reddest” pixel of each portion of theimage may be selected.

In an embodiment, a multi-layered network may be deployed to permitabstraction and generalization. The implementation of max-pooling layers520 and dropout layers may contribute to generalization of a model andthe ability to work on new, previously unseen faces.

In an embodiment, dimensionality reduction methods, such as principalcomponent analysis (PCA) and singular value decomposition (SVD), may beapplied to map a given problem space into a lower feature space. Inaddition, dimensionality reduction may be achieved by application offeatures inherent to one or more applied neural networks, allowing forsingle-step CNN processing and dimensionality reduction. Dimensionalityreduction methods may be applied to reduce the variables considered infacial analysis, allowing execution of methods directed to specificfacial features or expressions with reduced runtime, reduced resourcerequirements, or both. An example of a dimensional reduction is depictedwith respect to the flattening 530 and densification 540 shown in thediagram 500.

In the flattening 530 and densification 540 elements included in theexample diagram 500, a data feature is “flattened” 530 so thatcomponents of the data feature are uniformly accessible, rather thanretaining the nested or otherwise-structured formats which the data mayinclude, as received. Following “flattening” 530, the data is“densified” 540, reducing the presence of certain variables in the datafeature to enable further processing with greater efficiency. In anembodiment, the dimensional reduction process of flattening 530 anddensification 540 may be repeated multiple times, with the densifieddata feature being flattened and subsequently densified, again.

FIG. 6 shows an example block diagram of the system 130 implementedaccording to an embodiment. The system 130 includes a processingcircuitry 610 coupled to a memory 615, a storage 620, and a networkinterface 630. In an embodiment, the components of the system 130 may becommunicatively connected via a bus 640.

The processing circuitry 610 may be realized as one or more hardwarelogic components and circuits. For example, and without limitation,illustrative types of hardware logic components that can be used includefield programmable gate arrays (FPGAs), application-specific integratedcircuits (ASICs), application-specific standard products (ASSPs),system-on-a-chip systems (SOCs), general-purpose microprocessors,microcontrollers, graphics processing units (GPUs), tensor processingunits (TPUs), general-purpose microprocessors, microcontrollers, anddigital signal processors (DSPs), and the like, or any other hardwarelogic components that can perform calculations or other manipulations ofinformation.

The memory 615 may be volatile (e.g., RAM, etc.), non-volatile (e.g.,ROM, flash memory, etc.), or a combination thereof. In oneconfiguration, computer readable instructions to implement one or moreembodiments disclosed herein may be stored in the storage 620.

In another embodiment, the memory 615 is configured to store software.Software shall be construed broadly to mean any type of instructions,whether referred to as software, firmware, middleware, microcode,hardware description language, or otherwise. Instructions may includecode (e.g., in source code format, binary code format, executable codeformat, or any other suitable format of code). The instructions, whenexecuted by the one or more processors, cause the processing circuitry610 to perform the various processes described herein.

The storage 620 may be magnetic storage, optical storage, and the like,and may be realized, for example, as flash memory or another memorytechnology, CD-ROM, Digital Versatile Disks (DVDs), or any other mediumwhich can be used to store the desired information.

The network interface 630 allows the system 130 to communicate with theat least one various data sources (FIG. 1, 110 ). The embodimentsdescribed herein are not limited to the specific architectureillustrated in FIG. 6 , and other architectures may be equally usedwithout departing from the scope of the disclosed embodiments.

FIGS. 7A, 7B, and 7C are example images applicable to the disclosedembodiments. The various embodiments may be discussed with reference toFIGS. 7A, 7B, and 7C. FIG. 7A includes a picture of a woman at the ageof 85. The image of the woman in FIG. 7A includes a facial expressionwhich may be described as “sharp” and “focused,” indicating an absenceof dementia, considering the subject's age. In the example, theextraction of facial features, including the subject's eyes, as depictedin FIG. 7A, may be applicable to detection or diagnosis as a referencefacial profile, against which subsequent age-progressed profiles may becompared.

FIG. 7B includes a picture of the same woman at the age of 90. In theimage, FIG. 7B, the subject's appearance may be described, with respectto the eyes and facial expression, as “less focused” and “less sharp.”Where these observations may provide the basis for a dementia diagnosis,the condition may be observed by the comparison of FIGS. 7A and 7B toestablish facial feature changes over time. In an embodiment,age-progression images may be normalized against a standard facial agingtemplate to reduce or eliminate the presence of facial feature changesarising due to natural aging, allowing for detection ofcondition-related facial feature changes with greater accuracy andefficiency.

FIG. 7C includes a picture of the same woman at the age of 96. In theimage, FIG. 7C, the subject's appearance may be described, with respectto the eyes and facial expression, as including a “void gaze” andlacking notable facial expressions. Where these observations may providethe basis for diagnosis of advanced-stage dementia, the condition, andits development, may be observed with comparison to FIGS. 7A and 7B.

The embodiments disclosed herein can be utilized to detect otherconditions which may present a signature facial feature or features,such as Parkinson's disease, Alzheimer's disease, anxiety, depression,and the like. Where non-dementia conditions present signature orcharacteristic facial features, such as recognizable expressions,event-driven changes to expression or other facial features, orage-progressed facial changes, such as those shown in FIGS. 7A through7C, below, the disclosed embodiments may be appliable to the diagnosisof these conditions.

Where a condition is known to present a signature facial feature orexpression which is immediately apparent, requiring no showing of changefrom a baseline or reference facial feature, the condition may besuitable for diagnosis using methods similar or identical to thosedescribed herein. Certain conditions, including conditions manifestingextreme displays of emotions such as anger, fear, joy, and excitement,may be identified by characteristic facial features. In an embodiment,diagnosis of progression-independent conditions, for which no baselineor reference facial image is needed to accurately diagnose thecondition, may include extraction of facial features, as describedabove, in addition to comparison of the extracted features with one ormore condition or diagnosis datasets. In an embodiment, the condition ordiagnosis datasets used to diagnose progression-independent conditionsmay include one or more facial images, one or more extracted facialfeatures or sets of extracted facial features, other, like, data, andany combination thereof.

Further, the contents of the condition or diagnosis datasets may belabeled with one or more diseases, diagnoses, or conditions. Thecondition or diagnosis datasets may be stored in data repositoriesincluding, without limitation, the data store, 140, of FIG. 1 , above,various user or commercial devices, remote servers, other, like,repositories, and any combination thereof. Where a patient's extractedfacial features are similar or identical to those extracted facialfeatures included in the diagnosis or condition datasets, thecorresponding label or labels, as well as an associated confidencerating describing the similarity of the labeled facial features to thereceived facial features, may be returned in a manner similar oridentical to that described above.

Further, the embodiments disclosed herein may be applicable to thedetection of conditions for which a reference facial profile isrequired, but for which age-progression is not. As various conditions,such as jaundice and certain rashes, may be detected by observation ofvisual differences between a patient's current facial profile and areference or baseline profile, these conditions may be detected in asimilar manner. As above, facial features may be extracted from apatient's current and reference facial images and compared to isolatefacial feature differences arising between the instant image and thereference image. In an embodiment, the differences isolated may becompared to one or more facial difference datasets, the facialdifference datasets including extracted facial feature differencescorresponding with certain conditions, as well as associated data labelsindicating the condition with which the data is associated. Where apatient's isolated facial feature differences correspond with one ormore facial feature differences included in the dataset, the label orlabels corresponding with the indicated facial feature differences, aswell as an associated confidence rating, may be reported, as describedabove.

In addition, the disclosed embodiments may be further applicable to thedetection of conditions other than dementia which present characteristicfacial features across a given time period. Certain conditionsdisplaying characteristic time-progressed facial features may includeglaucoma, tooth and gum disease, various skin cancers, and other, like,conditions. As above, age-progressed conditions may be diagnosed bycomparison of the facial features extracted from a current image withthe facial features extracted from a previous image, and comparison ofthe differences between the features with a dataset includingtime-progressed facial feature differences and associated conditiondiagnosis labels. Time-progressed changes may be detected across one ormore sample images, examples of which may be noted with respect to FIGS.7A through 7C below, wherein a patient's facial features displaycharacteristic signs of a known condition across images captured atdifferent times. In the example figures, the patient's facial featuresdisplay time-progressed changes to the cheeks, mouth, eyes, eyebrows,chin, and jaw. In an embodiment, the application of time-progressedfacial feature diagnosis may include normalizing the detected facialfeature differences with one or more standard aging templates, allowingfacial changes due to standard aging to be separated from facial changesdue to a disease or condition.

The various embodiments disclosed herein can be implemented as hardware,firmware, software, or any combination thereof. Moreover, the softwareis preferably implemented as an application program tangibly embodied ona program storage unit or computer readable medium consisting of parts,or of certain devices and/or a combination of devices. The applicationprogram may be uploaded to, and executed by, a machine comprising anysuitable architecture. Preferably, the machine is implemented on acomputer platform having hardware such as one or more central processingunits (“CPUs”), a memory, and input/output interfaces. The computerplatform may also include an operating system and microinstruction code.The various processes and functions described herein may be either partof the microinstruction code or part of the application program, or anycombination thereof, which may be executed by a CPU, whether or not sucha computer or processor is explicitly shown. In addition, various otherperipheral units may be connected to the computer platform, such as anadditional data storage unit and a printing unit. Furthermore, anon-transitory computer readable medium is any computer readable mediumexcept for a transitory propagating signal.

As used herein, the phrase “at least one of” followed by a listing ofitems means that any of the listed items can be utilized individually,or any combination of two or more of the listed items can be utilized.For example, if a system is described as including “at least one of A,B, and C,” the system can include A alone; B alone; C alone; A and B incombination; B and C in combination; A and C in combination; or A, B,and C in combination.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the principlesof the disclosed embodiment and the concepts contributed by the inventorto furthering the art, and are to be construed as being withoutlimitation to such specifically recited examples and conditions.Moreover, all statements herein reciting principles, aspects, andembodiments of the disclosed embodiments, as well as specific examplesthereof, are intended to encompass both structural and functionalequivalents thereof. Additionally, it is intended that such equivalentsinclude both currently known equivalents as well as equivalentsdeveloped in the future, i.e., any elements developed that perform thesame function, regardless of structure.

What is claimed is:
 1. A method for generating a classifier to classifyfacial images for cognitive disorder in humans, comprising: receiving alabeled dataset including set of facial images, wherein each of thefacial image is labeled depending on whether it represents a cognitivedisorder condition; extracting, from the each of the facial image in theset of facial images, at least one learning facial feature indicative ofa cognitive disorder; feeding the at least one extracted facial featureto produce a machine learning trained model, wherein the at least oneextracted facial feature is identified to represent a human face; andgenerating a classifier based on the machine learning trained model,wherein the classifier is generated and ready when the trained modelincludes enough facial features processed by a machine learning model,wherein the classifier is configured to map a detection facial featurefrom a detection facial image to a score and output a plurality ofscores indicating a stage of a cognitive decline.
 2. The method of claim1, wherein the classifier is generated per type of the cognitivedisorder.
 3. The method of claim 1, wherein the cognitive disorderfurther includes illnesses including at least one of: dementia,Parkinson, and anxiety.
 4. The method of claim 1, wherein the machinelearning model is a deep neural network.
 5. The method of claim 1,further comprising: converting the each of the facial image in thereceived learning dataset into numerical descriptors reflectingattributes of the human face.
 6. A non-transitory computer readablemedium having stored thereon instructions for a processing circuitry toexecute a process for generating a classifier to classify facial imagesfor cognitive disorder in humans, the process comprising: receiving alabeled dataset including set of facial images, wherein each of thefacial image is labeled depending on whether it represents a cognitivedisorder condition; extracting, from the each of the facial image in theset of facial images, at least one learning facial feature indicative ofa cognitive disorder; feeding the at least one extracted facial featureto produce a machine learning trained model, wherein the at least oneextracted facial feature is identified to represent a human face; andgenerating a classifier based on the machine learning trained model,wherein the classifier is generated and ready when the trained modelincludes enough facial features processed by a machine learning model,wherein the classifier is configured to map a detection facial featurefrom a detection facial image to a score and output a plurality ofscores indicating a stage of a cognitive decline.
 7. A system forgenerating a classifier to classify facial images for cognitive disorderin humans, comprising: a processing circuitry; and a memory, the memorycontaining instructions that, when executed by the processing circuitry,configure the system to: receive a labeled dataset including set offacial images, wherein each of the facial image is labeled depending onwhether it represents a cognitive disorder condition; extract, from theeach of the facial image in the set of facial images, at least onelearning facial feature indicative of a cognitive disorder; and feed theat least one extracted facial feature into a to produce a machinelearning trained model, wherein the at least one extracted facialfeature is identified to represent a human face; and generate aclassifier based on the machine learning trained model, wherein theclassifier is generated and ready when the trained model includes enoughfacial features processed by a machine learning model, wherein theclassifier is configured to map a detection facial feature from adetection facial image to a score and output a plurality of scoresindicating a stage of a cognitive decline.
 8. The system of claim 7,wherein the classifier is generated per type of the cognitive disorder.9. The system of claim 7, wherein the cognitive disorder furtherincludes illnesses including at least one of: dementia, Parkinson, andanxiety.